Skip to content

[BUG] APIRequest doesn't support --max-http-header-size or something, leads to Header overflow #28873

New issue
Jump to bottom
New issue
Jump to bottom
Closed
Closed
[BUG] APIRequest doesn't support --max-http-header-size or something, leads to Header overflow#28873
@nonme

Description

@nonme
nonme
opened on Jan 5, 2024 · edited by nonme

System info

  • Playwright Version: 1.40.1
  • Operating System: All
  • Browser: Chromium (didn't test in others)

Source code

APIRequest.newContext().post(url, {
   data: requestBody,
   headers: {
   // Headers like Cookie:
   // To reproduce it should be enough to provide any large string of cookies
   }
 });

Here you usually include some headers - cookies, etc which you can get by using page.context() or set by yourself. If using headers longer than 8KB (or 16KB in latest node versions) you get Parse Error: Header Overflow preventing you from sending a request.

Expected

Setting node --max-http-header-size=SOME_BIG_NUMBER allows sending large headers for libraries like node-fetch etc.
Note that I'm actually using
node --max-http-header-size=5242880 -r ts-node/register ./src/main/main.ts
which still works fine with node-fetch, but not with APIRequest.

Actual

Playwright seems not to be affected by that option, and there's no other setting to allow sending large headers.

Also note that executing default fetch in browser via Page.evaluate of course works fine and this seems to be an issue related only to APIRequest.

Activity

roosalbe

roosalbe commented on Jan 5, 2024

@roosalbe
roosalbe
on Jan 5, 2024

I have got a similar issue with --insecure-http-parser that cannot be passed to Playwright

mxschmitt

mxschmitt commented on Jan 8, 2024

@mxschmitt
mxschmitt
on Jan 8, 2024
Member

Works for me, I was not able to reproduce:

// Invoke it with 'node --max-http-header-size=27000000 test.mjs'.
import playwright from 'playwright';
import http from 'http';

(async () => {
  const server = http.createServer((req, res) => {
    for (const [key, value] of Object.entries(req.headers))
      console.log(key, value.length);
    res.end();
  });
  await new Promise(resolve => server.listen(0, resolve));
  const apiRequestContext = await playwright.request.newContext();
  const response = await apiRequestContext.post(`http://localhost:${server.address().port}/`, {
    headers: {
      'content-type': 'application/json',
      // 16MB cookie
      'cookie': 'a'.repeat(16 * 1024 * 1024),
    },
  });
  console.log(response.status(), response.statusText());
  await apiRequestContext.dispose();
  server.close();
})();

@roosalbe please file a separate issue. But it should work as well.

nonme

nonme commented on Jan 11, 2024

@nonme
nonme
on Jan 11, 2024
Author

@mxschmitt I will look into it on weekends, thanks for your code snippet and a quick reply. Might be some compatibility issue between ts-node and Playwright as well, but the strange thing is that it's works for node-fetch but doesn't for Playwright. But I will look into it and report if this is not on Playwright's end.

mxschmitt

mxschmitt commented on Jan 16, 2024

@mxschmitt
mxschmitt
on Jan 16, 2024
Member

Closing in the meantime as part of our triage process. Feel free to re-file and reference this issue if you still encounter any issues.

mxschmitt
closed this as completedon Jan 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

      Participants

      @mxschmitt@nonme@roosalbe

      Issue actions
        [BUG] APIRequest doesn't support --max-http-header-size or something, leads to Header overflow · Issue #28873 · microsoft/playwright